Unbiased Multivariate Correlation Analysis

Correlation measures are a key element of statistics and machine learning, and essential for a wide range of data analysis tasks. Most existing correlation measures are for pairwise relationships, but real-world data can also exhibit complex multivariate correlations, involving three or more variables. We argue that multivariate correlation measures should be comparable, interpretable, scalable and unbiased. However, no existing measures satisfy all these requirements. In this paper, we propose an unbiased multivariate correlation measure, called UMC, which satisfies all the above criteria. UMC is a cumulative entropy based non-parametric multivariate correlation measure, which can capture both linear and non-linear correlations for groups of three or more variables. It employs a correction for chance using a statistical model of independence to address the issue of bias. UMC has high interpretability and we empirically show it outperforms state-of-the-art multivariate correlation measures in terms of statistical power, as well as for use in both subspace clustering and outlier detection tasks. Introduction Analysing correlations is a fundamental task in both statistics and machine learning. It has applications in many realworld learning tasks, e.g., feature selection (Brown et al., 2012), subspace search (Nguyen et al., 2013), causal inference (Bareinboim, Tian, and Pearl, 2014) and subspace clustering (Kriegel, Kröger, and Zimek, 2009). For the setting of continuous variables (as opposed to discrete), most existing correlation measures focus on pairwise relationships. For example, Pearson’s correlation coefficient detects bivariate linear correlations, and Maximal Information Coefficient (MIC) (Reshef et al., 2011) detects both linear and nonlinear bivariate correlations. However, real-world data often contains three or more variables which can exhibit multivariate (higher-order) correlations. If bivariate based measures are used to identify multivariate correlations, through pairwise aggregation, multivariate correlations can potentially be overlooked. For example, it has been shown that genes may reveal only a weak correlation with a disease when considered individually, while the correlation for a group of genes may be very strong (Zhang et al., 2008). Copyright c © 2017, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 2 3 4 5 6 7 8 9 10 Dimension 0.0 0.2 0.4 0.6 0.8 1.0 S c o re UDS for functional relationship UDS for independent variables UDS-r for functional relationship UDS-r for independent variables (a) UDS and UDS-r 2 3 4 5 6 7 8 9 10 Dimension 0.0 0.2 0.4 0.6 0.8 1.0